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Abstract 

This  paper  reports  on  an  experimental  study  where  178  student  subjects  without  formal 
training  in  contracting  issues  were  asked  to  accept  or  reject  each  of  20  clauses  of  a  software 
purchasing  contract.  The  subjects  used  a  Web-based  interface  to  accept  or  reject  clauses.  Of 
the  20  clauses  in  the  contract,  6  were  intentionally  deceitful,  in  the  sense  that  they  specified 
binding  obligations  that  made  it  unadvisable  to  accept  them  as  part  of  a  contract.  On  average, 
the  subjects  were  able  to  correctly  accept  approximately  1 1  out  of  14  non-deceitful  clauses. 
Somewhat  surprisingly,  the  subjects  were  able  to  correctly  reject  only  2  out  of  6  deceitful 
clauses.  The  study  also  suggests  that,  among  untrained  individuals,  those  who  are  older,  have 
more  general  work  experience,  and  have  above-average  scholastic  ability  are  the  ones  more 
likely  to  perform  well  in  contracting  tasks  under  conditions  similar  to  those  found  in  this  study 
(i.e.,  Web-based  contracting  conditions).  This  study’s  findings  provide  the  basis  for  a  strong  call 
for  more  and  better  training  of  contract  officers  in  the  DoD. 

KEYWORDS:  Electronic  Commerce,  Electronic  Trade,  Web-based  Contracts,  Software 
Purchasing,  Communication  Media,  Media  Naturalness 

Introduction 

Recently  there  have  been  renewed  calls  (see,  e.g.,  Friar,  2005)  for  more  and  better 
training  of  contract  officers  in  the  US  Department  of  Defense  (DoD).  Without  appropriate 
training,  serious  contractual  mistakes  may  be  made,  some  of  which  are  likely  to  place  the  DoD 
in  a  situation  where  it  is  legally  bound  to  abide  by  adverse  contract  clauses.  Given  the  large 
dollar  amounts  involved  in  many  of  the  DoD’s  contracts,  the  financial  consequences  of  such 
mistakes  can  be  significant. 

Several  technological  developments  in  the  last  20  years  led  to  a  significant  growth  in  the 
number  of  instances  of  situations  in  which  products  and  services  are  purchased  online.  Among 
those  technological  developments  were  the  emergence  of  the  Internet  in  the  early  1990s,  and 
the  advent  of  the  Web  in  the  mid-1990s  (Claycomb  et  al. ,  2005).  Most  online  transactions  that 
involve  the  purchase  of  products  and  services  go  through  the  reviewing,  completion,  and 
signing  of  a  Web-based  contract  (Atkins,  2003;  Backhouse  and  Cheng,  2000).  That  process 
usually  gravitates  around  the  acceptance  or  rejection  of  Web-based  contract  clauses. 

This  paper  empirically  illustrates  the  problems  associated  with  having  individuals  lacking 
proper  training  accept  or  reject  contract  clauses.  It  reports  on  an  experimental  study  where  a 
number  of  individuals  without  formal  training  in  contracting  issues  were  asked  to  accept  or  reject 
each  of  20  clauses  of  a  software  purchasing  contract.  The  clauses  were  developed  based  on  a 
several  sources,  including  existing  commercial  software  contracts  and  the  “Software  Legal 
Book”.  The  latter  is  a  study  conducted  by  the  Society  of  Information  Management  on  industry 
practices  associated  with  software  contracting. 

Research  method 

The  study  involved  178  student  subjects,  of  whom  approximately  57  percent  were 
males.  The  subjects’  ages  ranged  from  18  to  53  years  of  age,  with  a  mean  of  25  years.  Their 
work  experience  ranged  from  0  to  35  years,  with  a  mean  of  5.6  years.  Their  class  levels  ranged 
from  1  to  5;  the  levels  represented  in  this  study  were:  1  (freshman),  2  (sophomore),  3  (junior),  4 
(senior),  and  5  (graduate).  The  mean  class  level  of  the  subjects  was  3.3. 
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The  subjects  used  a  Web-based  interface  to  accept  or  reject  clauses.  Of  the  20  clauses 
in  the  contract,  6  were  intentionally  deceitful,  in  the  sense  that  they  specified  binding  obligations 
that  made  it  unadvisable  to  accept  them  as  part  of  a  contract.  Those  6  clauses  were  perceived 
as  obviously  deceitful  by  a  panel  of  experts  and  professionals  trained  on  contracting  issues  who 
were  asked  to  review  all  of  the  clauses.  From  a  practical  perspective,  correctly  rejecting 
deceitful  clauses  can  be  seen  as  more  critical  than  correctly  accepting  non-deceitful  clauses. 

The  measurement  model  included  three  latent  variables  measured  based  on 
perceptions.  These  latent  variables  were:  cognitive,  or  mental,  effort;  communication  ambiguity; 
and  dullness.  Perceived  cognitive  effort  is  sometimes  referred  to  here  as  COGEFF,  perceived 
communication  ambiguity  as  AMBIGU,  and  perceived  dullness  as  DULL.  Each  latent  variable 
was  measured  through  multiple  indicators.  The  question-statements  related  to  each  of  the 
indicators  are  listed  in  the  Appendix. 

The  relationship  between  latent  variables  and  other  variables  was  assessed  through 
structural  equation  modeling  employing  that  partial  least  squares  (PLS)  technique  (Chin  et  al. , 
1996;  Chin,  1998).  The  structural  equation  modeling  analyses  included  two  demographic 
variables  as  independent  and  intervening  variables,  namely  age  and  work  experience, 
respectively.  The  analyses  also  controlled  for  the  effects  of  several  demographic  variables, 
namely:  communication  medium  (text-based  or  video  clip-based  clauses  shown  on  a  Web 
browser),  gender,  scholastic  aptitude  (GPA),  and  class  level  (from  freshman  to  graduate). 


Results 


Table  1  shows  factor  loadings  obtained  through  a  factor  analysis.  The  extraction  method 
used  in  the  factor  analysis  was  principal  components,  and  the  rotation  method  employed  was 
varimax.  Shown  in  shaded  cells  are  the  loadings  for  the  indicators  that  were  conceptually 
expected  to  load  on  their  related  latent  constructs  (e.g.,  COGEFF1  ...  COGEFF4  were  expected 
to  load  on  COGEFF).  Also  shown  on  Table  1  are  Cronbach  alphas  for  each  of  the  latent 
constructs,  in  the  column  labeled  “Alpha”,  and  the  corresponding  composite  reliabilities,  in  the 
column  labeled  “CR”. 
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Table  1:  Factor  loadings  and  alpha  coefficients 


COGEFF 

AMBIGU 

DULL 

Alpha 

CR 

COGEFF1 

.76 

.39 

-.02 

.82 

.88 

COGEFF2 

.80 

.35 

.00 

COGEFF3 

.76 

.25 

.15 

COGEFF4 

.74 

.02 

.07 

AMBIGU1 

.36 

.76 

.18 

.87 

.92 

AMBIGU2 

.19 

.88 

.11 

AMBIGU3 

.23 

.85 

.18 

DULL1 

.09 

.13 

.91 

.91 

.94 

DULL2 

.07 

.14 

.90 

DULL3 

.03 

.13 

.92 

Notes: 


COGEFF  =  cognitive ,  or  mental,  effort 
AMBIGU  =  communication  ambiguity 
DULL  =  dullness 

Alpha  =  Cronbach  alpha  coefficient 

CR  =  composite  reliability  from  PLS  analysis 

The  convergent  validity  of  a  measurement  model  used  in  structural  equation  modeling 
can  be  assessed  based  on  the  comparison  of  indicator  loadings  expected  to  lead  on  each  of  the 
respective  latent  constructs  against  a  recommended  threshold  loading,  which  is  generally  .5 
(Hair  et  al.,  1987).  For  this  study,  the  indicator  loadings  in  question  are  the  ones  shown  in  the 
shaded  cells  in  Table  1.  They  range  from  .74  to  .92,  suggesting  that  the  measurement  model 
presents  an  appropriate  level  of  convergent  validity. 

Another  important  attribute  of  a  measurement  model  is  its  reliability,  which  can  be 
assessed  through  measures  such  as  Cronbach  alpha  and  composite  reliability  coefficients.  As 
with  convergent  validity  analysis,  the  determination  as  to  whether  a  measurement  model  has 
good  reliability  hinges  on  the  comparison  of  reliability  measures  with  a  recommended  threshold. 
That  threshold  is  generally  .7  (Nunnaly,  1978).  As  it  can  be  seen  in  the  columns  labeled  “Alpha” 
and  “CR”  of  Table  1,  all  Cronbach  alpha  and  composite  reliability  coefficients  are  higher  than 
.82.  This  leads  to  the  conclusion  that  the  measurement  model  employed  presents  an 
appropriate  level  of  reliability. 
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Table  2  shows  the  one-on-one  correlations  between  the  latent  constructs,  as  well  as  the 
average  variances  extracted  for  each  latent  construct  -  the  latter  are  shown  on  the  diagonal, 
within  parentheses.  Also  shown  are  the  means  and  standard  deviations  for  each  of  the 
constructs.  The  correlation  coefficients  shown  were  calculated  based  on  Pearson’s  method,  and 
refer  to  bivariate  rather  than  partial  correlation  estimations.  All  of  the  correlations  shown  are 
significant  at  the  .05  level  (marked  with  “*”)  or  at  the  .01  level  (marked  with  “**”). 

Table  2:  Correlations,  AVEs,  means  and  standard  deviations 


COGEFF 

AMBIGU 

DULL 

COGEFF 

(.65) 

AMBIGU 

.58** 

(.79) 

DULL 

.17* 

.32** 

(.85) 

Mean 

4.15 

4.14 

3.96 

SD 

1.16 

1.20 

1.28 

Notes: 


Coefficients  shown  are  Pearson  correlations,  and  average  variances  extracted  (diagonal) 

COGEFF  =  cognitive,  or  mental,  effort 

AMBIGU  -  communication  ambiguity 

DULL  =  dullness 

SD  =  standard  deviation 

*  =  correlation  significant  at  the  .05  level 

**  =  correlation  significant  at  the  .01  level 

One  final  attribute  that  is  often  assessed  for  a  measurement  model  based  on  latent 
constructs  is  the  discriminant  validity  of  the  model.  This  attribute  can  be  assessed  based  on  the 
comparison  of  the  correlations  between  the  latent  constructs  and  their  individual  average 
variances  extracted  (Fornell  &  Larcker,  1981).  Here,  a  conservative  criterion  that  can  be  used  to 
assess  discriminant  validity  is  to  check  whether  the  average  variance  extracted  for  each  latent 
construct  is  higher  than  any  of  the  correlations  involving  the  construct  in  question.  As  it  can  be 
seen  from  Table  2,  all  of  the  average  variances  extracted  (shown  on  the  diagonal)  are  higher 
than  the  correlations  displayed  below  them  or  to  their  left.  Therefore,  it  can  be  concluded  that 
the  measurement  model  presents  an  appropriate  level  of  discriminant  validity. 

Now  that  it  has  been  established  that  the  measurement  model  has  appropriate  validity 
and  reliability,  it  is  possible  to  conduct  a  meaningful  structural  equation  modeling  analysis  of  the 
relationships  between  several  of  its  variables.  The  results  of  one  such  analysis  are  shown  in 
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Figure  1.  There,  several  variables  are  shown  within  ovals.  Curved  arrows  connecting  variables 
depict  causal  relationships  tested  through  the  structural  equation  modeling  analysis.  Dotted 
arrows  indicate  relationships  that  are  not  statistically  significant;  full  arrows  indicate  statistically 
significant  ones. 

The  p  coefficients  near  the  full  arrows  in  Figure  1  refer  to  the  partial  correlations 
associated  with  the  significant  relationships.  Dotted  arrows  have  the  letters  “NS”  (not  significant) 
shown  instead  of  p  coefficients.  The  p  coefficients  displayed  are  followed  by  the  symbols  “*”  or 
which  indicate  significance  levels  -  .05  or  .01 ,  respectively.  The  R2  coefficients  shown 
undereach  endogenous  (i.e.,  dependent)  variable  indicate  the  percentage  of  explained  variance 
provided  by  the  model  for  that  particular  variable. 
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Figure  1:  Structural  equation  model  and  estimated  parameters 


Notes: 

Good  accepts  =  number  of  non-deceitful  clauses  accepted 

Good  rejects  =  number  of  deceitful  clauses  rejected 

(3  =  partial  correlation  coefficient  associated  with  a  causal  link  in  the  model 

R2  =  variance  explained  by  the  model  for  a  particular  endogenous  variable 

*  =  causal  link  significant  at  the  .05  level 

**  =  causal  link  significant  at  the  .01  level 

NS  =  causal  link  not  significant 
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As  it  can  be  seen  from  Figure  1,  age  appears  to  have  a  strong  relationship  with  work 
experience  (P=.81 ,  P<.01 ),  which  is  to  be  expected  since  older  individuals  usually  have  more 
work-related  experience  than  younger  ones.  The  relationship  between  age  and  perceived 
dullness  was  also  significant,  although  negative  (P=-.27,  P<.01).  That  is,  older  individuals 
seemed  to  perceive  the  task  of  accepting  or  rejecting  contract  clauses  as  less  boring  than 
younger  individuals.  Age  seemed  to  also  be  significantly  and  negatively  related  to  perceived 
communication  ambiguity  (P=-.24,  P<.01),  which  means  that  older  individuals  seemed  to 
perceive  the  contract  clauses  as  less  confusing  than  younger  individuals.  Age  was  not 
significantly  related  to  perceived  cognitive  effort. 

Work  experience  appears  to  have  a  significant  and  negative  relationship  with  perceived 
cognitive  effort  (P=-.28,  P<.05),  which  suggests  that  individuals  with  more  work  experience 
tended  to  perceive  the  task  of  accepting  or  rejecting  clauses  as  less  complex  and  mentally 
demanding  than  individuals  with  less  work  experience.  The  relationship  between  work 
experience  and  perceived  communication  ambiguity  was  also  significant,  although  positive 
(P=.16,  P<.05),  which  means  that  individuals  with  more  work  experience  were  inclined  to 
perceive  the  contract  clauses  as  more  confusing  than  younger  individuals.  Work  experience 
was  not  significantly  related  to  perceived  dullness. 

The  degree  of  cognitive  effort  perceived  by  the  subjects  seems  to  have  a  strong  and 
positive  relationship  with  perceived  communication  ambiguity  (P=.60,  P<.01).  That  is,  individuals 
who  perceived  the  task  of  accepting  or  rejecting  clauses  as  more  complex  and  mentally 
demanding  also  perceived  the  contract  clauses  as  more  confusing.  Perceived  cognitive  effort 
was  not  significantly  related  to  perceived  dullness. 

Perceived  communication  ambiguity  appears  to  be  significantly  related  to  perceived 
dullness  (P=.30,  P<.01),  which  essentially  means  that  individuals  who  perceived  the  contract 
clauses  as  more  confusing  also  perceived  the  task  of  accepting  or  rejecting  contract  clauses  as 
more  boring.  There  seemed  to  also  be  a  significant  relationship  between  perceived 
communication  ambiguity  and  number  of  appropriate  rejections  of  deceitful  clauses  (P=.15, 
P<.05),  noted  as  “good  rejects”  in  Figure  1.  In  other  words,  individuals  who  perceived  the 
contract  clauses  as  more  confusing  also  did  better  in  terms  of  rejecting  deceitful  clauses. 
Perceived  communication  ambiguity  was  not  significantly  related  to  the  number  of  appropriate 
acceptances  of  non-deceitful  clauses. 

Perceived  dullness  seems  to  be  significantly  and  negatively  related  to  the  number  of 
appropriate  acceptances  of  non-deceitful  clauses  (P=-.28,  P<.01).  This  suggests  that  individuals 
who  perceived  the  task  of  accepting  or  rejecting  contract  clauses  as  more  boring  also  did  worse 
in  terms  of  appropriately  accepting  non-deceitful  clauses.  Perceived  dullness  was  not 
significantly  related  to  the  number  of  appropriate  rejections  of  deceitful  clauses. 

Not  shown  on  Figure  1  are  the  control  variables,  which  were  included  in  the  model  as 
independent  variables  pointing  at  the  two  main  dependent  variables  of  the  model  -  namely  good 
rejections  and  acceptances.  Those  control  variables  were  communication  medium  (text-based 
or  video  clip-based  clauses  shown  on  a  Web  browser),  gender,  scholastic  aptitude  (GPA),  and 
class  level  (from  freshman  to  graduate).  None  of  those  variables  had  a  statistically  significant 
effect  on  either  of  the  two  main  dependent  variables  of  the  model. 

Figure  2  shows  a  bar  chart  depicting  the  average  numbers  of  appropriate  rejections  of 
deceitful  clauses  (i.e.,  “good  rejects”),  and  of  appropriate  acceptances  of  non-deceitful  clauses 
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(i.e. ,  “good  accepts”).  Since  each  clause  could  either  be  rejected  or  accepted,  and  there  were  6 
deceitful  clauses,  the  average  number  of  rejections  of  deceitful  clauses  obtained  by  chance 
would  be  6  divided  by  2,  or  3.  That  is,  if  all  individuals  had  accepted  or  rejected  clauses 
randomly,  the  average  number  of  good  rejects  would  be  3.  Similarly,  since  the  number  of  non¬ 
deceitful  clauses  was  14,  the  average  number  of  good  accepts  obtained  by  chance  would  be  7. 

Figure  2:  Average  numbers  of  good  rejections  and  acceptances 


Notes: 

Good  rejects  =  number  of  deceitful  clauses  rejected 
Good  rejects  obtained  by  chance  =  3 
Good  accepts  =  number  of  non-deceitful  clauses  accepted 
Good  accepts  obtained  by  chance  =  7 

As  it  can  be  seen  from  Figure  2,  the  individuals  participating  in  this  study  generally  did 
worse  than  chance  in  terms  of  appropriately  rejecting  deceitful  clauses,  as  the  average  number 
of  good  rejects  was  a  little  over  2  (to  be  more  precise,  it  was  2.07).  The  standard  deviation  for 
good  rejects  was  1 .41 .  Therefore,  the  average  number  of  good  rejects  was  approximately  two 
thirds  of  a  standard  deviation  lower  than  the  number  of  chance  good  rejects;  a  difference  large 
enough  to  be  considered  statistically  significant  given  the  sample  size  and  data  distribution  in 
this  study. 

It  can  also  be  inferred  from  Figure  2  that  the  individuals  did  better  than  chance  in  terms 
of  good  accepts,  or  in  terms  of  appropriately  accepting  non-deceitful  clauses,  as  the  average 
number  of  good  accepts  was  a  little  over  11(11 .38,  to  be  more  accurate).  The  standard 
deviation  for  good  accepts  was  2.44,  which  means  that  the  individuals’  performance  in  terms  of 
accepting  non-deceitful  clauses  was  1.79  standard  deviations  above  the  chance  performance. 
This  is  not  only  statistically  significant,  but  also  a  relatively  large  difference  in  statistical  terms. 
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Discussion  and  conclusion 


On  average,  the  subjects  were  able  to  correctly  accept  approximately  1 1  out  of  14  non¬ 
deceitful  clauses.  Somewhat  surprisingly,  the  subjects  were  able  to  correctly  reject  only  about  2 
out  of  6  deceitful  clauses.  It  is  no  exaggeration  to  say  that  a  monkey  trained  to  mindlessly 
accept  or  reject  those  deceitful  clauses  would  have  performed  better  than  this  study’s  subjects, 
since  that  monkey  would  have  on  average  rejected  50  percent  (or  3)  of  the  6  deceitful  clauses. 
That  is,  on  average  the  subjects  performed  worse  than  chance  in  terms  of  rejecting  deceitful 
clauses. 

The  study  also  looked  into  the  relationship  between  the  ability  to  correctly  accept  or 
reject  clauses  and  three  key  perceptual  latent  variables:  cognitive  (or  mental)  effort, 
communication  ambiguity  (or  confusion),  and  dullness  (or  boredom).  The  degree  of  perceived 
communication  ambiguity  was,  somewhat  surprisingly,  positively  correlated  with  the  ability  to 
correctly  reject  deceitful  clauses  (perhaps  because  the  sense  of  ambiguity  led  subjects  to  be 
more  alert  to  deceit).  The  degree  of  perceived  dullness  experienced  by  the  study  subjects  was 
negatively  correlated  with  the  ability  to  correctly  accept  non-deceitful  clauses  (i.e.,  bored 
individuals  seemed  more  likely  to  reject  acceptable  clauses).  Perceived  cognitive  effort  was 
strongly  and  positively  related  to  perceived  communication  ambiguity  (i.e.,  mentally  drained 
individuals  seemed  more  likely  to  fell  confused  by  the  clauses),  but  not  to  perceived  dullness. 

Both  demographic  variables  included  in  the  model  as  independent  and  intervening 
variables,  namely  age  and  work  experience,  had  statistically  significant  effects  on  other 
variables.  The  study  suggests  that  age  is  negatively  related  to  perceived  communication 
ambiguity  and  dullness  (i.e.,  older  individuals  experienced  less  confusion  and  boredom  while 
going  through  contract  clauses).  It  also  suggests,  somewhat  intuitively,  that  work  experience  is 
negatively  related  to  perceived  cognitive  effort  (i.e.,  subjects  with  more  work  experience  felt  less 
mentally  “drained”  by  the  task).  Additionally,  the  study  suggests  that  age  is  positively  related  to 
work  experience,  as  one  would  expect. 

This  study’s  findings  provide  the  basis  for  a  strong  call  for  more  and  better  training  of 
contract  officers  in  the  DoD.  Those  findings  also  suggest  that,  among  untrained  individuals, 
those  who  are  older  and  have  more  general  work  experience  are  the  ones  more  likely  to 
perform  well  in  contracting  tasks  under  conditions  similar  to  those  found  in  this  study  (i.e.,  Web- 
based  contracting  conditions).  Finally,  this  study  suggests  exciting  new  avenues  for  research  on 
contracting  issues,  particularly  in  connection  with  deceit  identification  in  contracts.  Untrained 
individuals  seem  fairly  unable  to  identify  deceit,  even  in  cases  where  trained  individuals  would 
perceive  it  as  obvious. 
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